This section introduces aspects that may help facilitate a better understanding of the disclosure. Accordingly, these statements are to be read in this light and are not to be understood as admissions about what is or is not prior art.
Human capabilities enabled by visual, aural, tactile, kinesthetic, and spatial perceptions are underutilized by digital media for artifact creation. To cooperate with an increasingly growing world of digital manufacturing/fabricating, humans have been forced to use multi-step, tedious, and cumbersome interactions with the associated virtual design and display systems. In such processes, natural modes of thinking and communication are fractured because of: (1) switching between tangible instruments such as measurement and input devices like keyboards and mice, and (2) limitations in visual perception of virtual representations of physical artifacts. These discontinuities in interactions are a result of background representations and modalities for information exchange used by these machines and devices. For example, the design environment is typically implemented through a 2D Windows-Icons-Menus-Pointers (WIMP) manner which requires extensive training and inhibit users ability to create virtual 3D shapes in a simple manner. Moreover, such virtual design environment is isolated from the design context which results in the visual perception of virtual contents separates from the physical artifacts and the surrounding environment. The physical environment often serves as a means for inspiring, contextualizing, and guiding the designer's thought process for expressing creative ideas. In early design processes, objects are frequently used as references to explore the space of novel designs.
Emerging mobile devices further led to a distributive change in the way users create, manipulate, and share digital information. Methods for using mobile devices for design environments and interfaces have been recently proposed. Although, these works leverage the ubiquity of mobile devices, they haven't addressed the fragmented virtual design environment and the physical design context. Recent works have shown that through-the-screen Augmented Reality (AR) and Mixed Reality (MR) can play a vital role in bridging the gap between the physical and digital worlds for creative expression of ideas. However, most of these approaches use the physical environment mainly as a dormant container of digital artifacts rather than as a source of inspiration for facilitating quick digital prototyping for design ideation. Therefore, improvements are needed in the field.